Tversky as a Loss Function for Highly Unbalanced Image Segmentation using 3D Fully Convolutional Deep Networks

Fully convolutional deep neural networks have been asserted to be fast and precise frameworks with great potential in image segmentation. One of the major challenges in utilizing such networks is data imbalance, which is especially restraining in medical imaging applications such as lesion segmentation where lesion class voxels are often much less than non-lesion voxels. A trained network with unbalanced data may make predictions toward high precision and low recall (sensitivity), being severely biased towards the non-lesion class which is particularly undesired in medical applications where false negatives are actually more important than false positives. Several methods have been proposed to deal with this problem including balanced sampling, two step training, sample re-weighting, and similarity loss functions. In this paper, we propose a generalized loss function based on the Tversky index to mitigate the issue of data imbalance and achieve much better trade-off between precision and recall in training 3D fully convolutional deep neural networks. Moreover, we extend our preliminary work on using Tversky loss function for U-net to a patch-wise 3D densely connected network, where we use overlapping image patches for intrinsic and extrinsic data augmentation. To this end, we propose a patch prediction fusion strategy based on B-spline weighted soft voting to take into account the uncertainty of prediction in patch borders. The lesion segmentation results obtained from our patch-wise 3D densely connected network are superior to the recently reported results in the literature on multiple sclerosis lesion segmentation on magnetic resonance imaging dataset, namely MSSEG 2016, in which we obtained average Dice coefficient of 69.8%. Significant improvement in F1 and F2 scores and the area under the precision-recall curve was achieved in test using the Tversky loss layer and via our 3D patch prediction fusion method. Based on these results we suggest Tversky loss function as a generalization of the Dice similarity coefficient and Fβ scores to effectively train deep neural networks.